Recommending competitive learning objects

ABSTRACT

A method and apparatus for competitive learning objects in a learning environment is provided. Based on the success that a student has with a particular learning item, a learning recommendation is generated for another student.

CROSS-REFERENCE TO RELATED APPLICATIONS Benefit Claim

This application is a Continuation of Non-Provisional application Ser.No. 13/007,177 filed Jan. 14, 2011, which claims the benefit ofProvisional Appln. 61/295,635, filed Jan. 15, 2010 and ProvisionalAppln. 61/334,158, filed May 12, 2010, the entire contents of all ofwhich are hereby incorporated by reference as if fully set forth herein,under 35 U.S.C. §119(e). This application is related to the followingapplications: (1) application 13/007,147, entitled “DYNAMICALLYRECOMMENDING LEARNING CONTENT,” Attorney Docket Number 60201-0043, filedon Jan. 14, 2011, the entire contents of which is hereby incorporated byreference as if fully set forth herein; and (2) application 13/007,166,entitled “FACILITATING TARGETED INTERACTION IN A NETWORKED LEARNINGENVIRONMENT,” Attorney Docket Number 60201-0052, filed on Jan. 14, 2011,the entire contents of which is hereby incorporated by reference as iffully set forth herein. The applicant(s) hereby rescind any disclaimerof claim scope in the parent application(s) or the prosecution historythereof and advise the USPTO that the claims in this application may bebroader than any claim in the parent application(s).

FIELD OF THE INVENTION

The present invention relates to learning management systems. Inparticular, the present invention relates to platforms forindividualized learning.

BACKGROUND

Intelligent learning systems are systems that attempt to assist studentsin achieving specific learning goals. To date, these systems have mainlyused a computerized teaching approach that minors the approach taken inbrick-and-mortar classrooms. Each student is presented with the samelecture, content, and assessment, regardless of learning style,intelligence, or cognitive characteristics.

Advances in intelligent learning systems have been limited to approachessuch as “adaptive learning.” These approaches are usually applied tologic-based topics such as mathematics, where the content that is servedto each student is based on a pre-determined course-specific decisiontree that is hard-coded into the system. If a first student and a secondstudent each fail the same assessment by missing the same questions,both students will be presented with the same remedial materials asdictated by the decision tree.

Online courses are examples of “containers” that may employ adaptivelearning technology to achieve a specific goal. For any given container,the adaptive learning technology used by the container is largelyself-contained. That is, the adaptive learning technology employed by acontainer is programmed for a singular unchanging goal associated withthe container.

For example, an adaptive learning tool may be designed to teach astudent a course on the fundamentals of calculus. The designer of thetool will assume that the student possesses the foundational knowledgeof mathematics required to begin the course, but the tool may provide acertain amount of “review” information as a means of calibration. Inaddition, the tool will not take into consideration the goals of anyother course in which the student may be engaged. Instead, the tool willbe designed to help the student achieve a particular level ofproficiency in calculus. Once that level of efficiency is obtained bythe student, the tool becomes useless. While data, such as assessmentscores, may be saved, the core logic of the adaptive learning toolprovides no additional benefit to the student unless the student decidesto re-take the course or a portion of the course.

The illusion of adaptivity in “adaptive learning” tools is achieved byproviding a dynamic experience for the student. This experience is basedon the relationship between the assessment scores of the student and thepre-programmed hierarchy included in the tool. However, existing toolsdo not actually “adapt” to the student. Instead, by performing in aparticular way, the student merely traverses down a pre-existing paththrough the tool's hierarchy.

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1 is a block diagram illustrating a learning management platform onwhich an embodiment may be implemented.

FIG. 2 is a block diagram illustrating a skills hierarchy structure thatmay be used in an embodiment.

FIG. 3 is a block diagram illustrating a learning object in anembodiment.

FIG. 4 illustrates a content feedback interface in an embodiment.

FIG. 5 illustrates a computer system upon which an embodiment may beimplemented.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present invention. It will be apparent, however,that the present invention may be practiced without these specificdetails. In other instances, well-known structures and devices are shownin block diagram form in order to avoid unnecessarily obscuring thepresent invention.

General Overview

It is common for students to take courses that provide too muchinformation that the student already knows. On the other hand, it isalso common for a student to complete a course in a sequence, only to beleft behind in the next course in the sequence because the next courseassumes knowledge that was not sufficiently covered in the earliercourse. Students may also be required to take certain courses tocomplete a degree program, even if only a small portion of the skillslearned in those courses pertain to the student's degree program. Oneway to ensure that students get the most out of their educationalexperiences is to personalize the educational experiences.

By using a learning management platform that employs the techniquesdescribed hereafter, students can receive personalized learningrecommendations that identify specific content or learning objects thatpertain to each individual student. In an embodiment, each learningobject is associated with an individual skill and content associatedwith that skill, and a single course (i.e., Math 101) is comprised ofmany learning objects. Learning objects are organized in a hierarchythat is based on the skills associated with the learning objects.Learning objects can be made to compete with one another for a spot inthe hierarchy so that the “best” learning object can be recommended moreoften.

In an embodiment, a learning management platform generates learningrecommendations for students. The learning management platformimplements multiple learning models and instructional strategies toguide a student throughout that student's academic journey in a wayconsistent with the student's cognitive characteristics and otherattributes relevant to learning. The platform treats a student's journeyas a life-long continuum and provides a set of powerful capabilities toserve the information necessary to support the student over time, evenas the student's motivations and goals change. The learning managementplatform provides the “right outcome” for each student for a broad-basedcurriculum, and ultimately for each student's life-long learning goals.

If a student already possesses a particular skill, that student mayproceed to learning a more advanced skill in the hierarchy, in anembodiment. If a student attempts to complete a learning object and doesnot succeed, the recommendation provided for that student includes aremedial learning objet to help prepare the student for another attemptat the first learning object. As different students attempt remediationin this way, the learning management platform learns which objects aregood remediators for other objects.

In an embodiment, the student interacts with the learning managementplatform through a “learning space” platform, which may be a web-basedapplication or an application being executed on one or more devices orcomputers associated with the student. The learning space platformdefines the student's experience, and provides feedback to the learningmanagement platform. The learning space platform makes a request to alearning experience engine. The request includes student profileinformation. Based on information associated with the student profileinformation, the learning experience engine provides an individualizedlearning recommendation. For example, an English major may receive arecommendation to learn from a particular learning object associatedwith the skill “using prepositions properly”, using particular contentassociated with that object, such as a video or audio lecture associatedwith the skill.

In an embodiment, user profiles are compared with one another toidentify students that are similar to one another. These similaritiesmay help the learning management platform decide the best recommendationfor a student. For example, if a student is similar to another studentthat successfully learned a particular skill using a particular learningobject, then the particular learning object may be recommended. Inaddition, user profiles are be used to determine which users shouldinteract with one another in online collaborative learning sessions.

Recommendations may also be based on the learning context of thestudent. In an embodiment, environmental and/or emotional circumstancesassociated with the student may alter the learning recommendation. Forexample, a learning object that requires a student to draw pictures maynot be appropriate for a student using a mobile device with a smallscreen while riding a train.

In another embodiment, the individualized learning recommendation isbased at least in part on a learning skills hierarchy. In an embodiment,the learning skills hierarchy is a hierarchical multidimensionaldirected graph that has, as nodes, learning objects associated withskills, assessments, and content.

Individualized Learning Recommendations

As used herein, the term “learning recommendation” refers toinformation, provided to a student or a device associated with astudent, which provides educational direction. For example, a user thatcompletes the course “Math 101” may be advised by a learningrecommendation to begin the course “Math 102.” Learning recommendationsneed not be limited to course recommendations, however. Instead, skills,content, tools, and activities may be recommended to a student with thegoal of furthering the education of the student.

An individualized learning recommendation is a learning recommendationthat is based on individual attributes of the student. For example, anindividualized learning recommendation may take into account the type oflearner a student classifies himself as. A student may receiverecommendations to watch educational videos if the student identifieshimself as a “visual” learner. However, if the same student performspoorly on assessments after studying with video content, but performswell on assessments after reading materials, then the learning platformmay determine that the student's belief that he is a visual learner maybe incorrect. In this case, future individualized learningrecommendations may not include video content.

Individualized learning recommendations may be based on many types ofinformation related to the student, such as past performance, interests,major, and various demographic information. These attributes may becompared with the attributes of other students that have had similareducational needs, and an individualized learning recommendation may bebased on the success of similar students. For example, a first studentmay be required to learn integration as part of a course in Calculus.Different students, with similar attributes as the first student, thathave successfully learned integration may provide insight into whichcontent, tools, and activities will help the first student be successfulin learning integration.

Structural Overview

FIG. 1 is a block diagram illustrating a learning management platform100, according to an embodiment of the invention. Learning managementplatform 100 generally represents a set of one or more computerprograms, and associated resources, configured to manage educationaldata and information about students, provide learning recommendations tostudents, and use information gathered from analyzing studentinteraction with the system to increase the effectiveness of futurelearning recommendations. Learning management platform 100 facilitatesthe delivery of information based on learning theories, models, andstrategies. Learning management platform 100 includes logic thatfacilitates communication between its various components.

In the illustrated embodiment, learning management platform 100 includesa learning experience engine 110, a learning space platform 120, alearning content manager 130, a learning tools manager 140, a dataanalysis engine 150, a knowledge base 160, a personal cognitive DNAmanager 170, a skills hierarchy manager 180, and learning system modules190A-C. Each of these components of platform 100 shall be described ingreater detail hereafter.

The Learning Experience Engine

Learning experience engine 110 generally represents a decision-makingengine that interacts with all other components of learning managementplatform 100 and uses information gathered from these components toprovide the best learning recommendation possible to students thatinteract with learning management platform 100. According to oneembodiment, learning experience engine 110 includes learningrecommendation logic configured to provide individualized learningrecommendations based on information gathered from other elements of thelearning management platform 100, such as knowledge base 160 andpersonal cognitive DNA manager 170.

Unlike conventional “adaptive learning” systems, learning experienceengine 110 makes learning recommendations that are not based merely ontraversal of a predetermined path that is based only on the student'sdegree program or class. Rather, learning experience engine 110 takesinto consideration transient and non-transient profile attributes ofeach student. A transient profile attribute is any attribute thatchanges with relatively high frequency. A transient attribute may, forexample, change simultaneously with environmental, physical or emotionalcircumstances associated with the corresponding user. Thus, the currentlocation of a student would typically constitute a transient profileattribute. In contrast, a non-transient profile attribute is anyattribute that changes rarely, if ever. Non-transient profile attributesinclude, for example, the birth date or home address of a student.

As an example of how learning experience engine 110 may make use ofprofile attributes, consider a situation in which a student may want tolearn a skill, such as how to use gerunds in a sentence, while travelingon an airplane. Learning experience engine 110 may ask for the expectedarrival time to determine how much time the student has left on theplane. Then, taking into account how much time the student has left onthe plane, as well as attributes such as that student's learning styleand habits, as well as the type of device that the student is using,learning experience engine 110 selects appropriate content, such asaudio/video content and text content, for deliver to the student'sdevice.

As the data that drives the decision-making process of the learningexperience engine 110 evolves, the decisions made by learning experienceengine 100 become more accurate. Students may receive hundreds oflearning recommendations over time. As students perform activities andtake an assessments associated with recommendations, data associatedwith each student is updated to reflect the types of activities thatwork well for each student, the strengths and weaknesses of the student,and other useful education-related attributes of the students.

As used herein, the term “education-related attributes” refers to anyattributes that relate to a student's learning history, goals orabilities. As shall be described in greater detail below,education-related attributes may include non-transient attributes, suchas a student's prior classes and grades, and transient attributes suchas a student's current mood.

As more data is collected, patterns emerge, and learning experienceengine 110 can provide individualized learning recommendations with ahigh degree of confidence in the expected success of each student. Forexample, it may become clear that a particular student performs poorlywhen he tries to learn skills using only audio content, even though thatstudent has expressed a preference for audio content. In this case,learning experience engine 110 may subsequently require content otherthan audio content to be delivered to the user, instead of or inaddition to audio content.

Other types of data that may be used in determining individualizedlearning recommendations are discussed herein. Rules defining the use ofthis data may be used to configure the learning experience engine 110.In addition, weights, confidence metrics, and other measurements of dataimportance and accuracy may be assigned to any type of data discussedherein, and these measurements may be taken into consideration in therules that define the use of the data by learning experience engine 100.

In one embodiment, learning experience engine 110 communicates andshares information with other elements of the learning managementplatform 100. For example, data analysis engine 150 may not be directlycommunicatively coupled to skills hierarchy manager 180 in anembodiment. In an embodiment where data analysis engine 150 and skillshierarchy manager 180 do not directly communicate with each other,communication between data analysis engine 150 and skills hierarchymanager 180 may nevertheless be carried out using learning experienceengine 110 as an intermediary in the communication operation.

In other embodiments, other elements of the learning management platform100 may be directly communicatively coupled to one another, andcommunication does not require the use of the learning experience engine110 as an intermediary. For example, personal cognitive DNA manager 170may directly communicate with knowledge base 160.

The Learning Space Platform

Learning space platform 120 represents the user interface that thestudent sees when interacting with learning management platform 100.Learning space platform 120 also includes logic that is specific to thedevice on which learning space platform 120 resides. Learning spaceplatform 120 includes logic configured to interact with other elementsof learning management platform 100. For example, learning spaceplatform 120 may receive a learning recommendation from learningexperience engine 110, and based on this learning recommendation,learning space platform 120 may request content from learning contentmanager 130 and tools from learning tools manager 140. While the toolsthemselves can be “plugged into” the Learning Space using variousinteroperability standards in existence (such as IMS's Learning ToolsInteroperability standard or the web's Open Social standard) and singlesign-on techniques, the learning space platform 120 provides theexperience recommended to complete the tasks a student needs to masterin order to meet their next outcome by facilitating the delivery oflearning content using appropriate tools.

Learning space platform 120 resides on a client computing device, in anembodiment. A client computing device includes any device capable ofpresenting a user with learning information, such as a personalcomputer, mobile computing device, set-top box, or network basedappliance. In other embodiments, learning space platform 120 resides ona terminal server, web server, or any other remote location that allowsa user to interact with learning space platform 120. For example,learning space platform 120 may be a web-based interface included inlearning experience engine 110.

Learning space platform 120 is used to make “local” decisions about thestudent experience, in an embodiment. For example, learning spaceplatform 120 may be an iPhone application detects the location of thestudent or asks for feedback from the student, such as feedback relatedto the student's mood. The location data and mood data may then be usedto determine the best learning experience. Local decisions may also bebased on the screen size or other attributes of the device on which thelearning space platform 120 resides. For example, learning experienceengine 110 may deliver a variety of learning content to the student.After the content has been received, learning space platform 120 decideswhich content to display, and how to display it, taking intoconsideration screen size, stability of Internet connection, or localpreferences set by the student. In an embodiment, the decisionsdiscussed above may be made by learning experience engine 110.

In addition, learning space platform 120 may provide detailed user andtime-specific transient data to learning experience engine 110, in anembodiment. For example, the current location of the user may beprovided by learning space platform 120 to learning experience engine110, which stores the data and uses it as input for learningrecommendation decisions. As another example, the learning spaceplatform 120 may communicate the current speed at which the user ismoving, thereby allowing the learning experience engine 110 to makerecommendations based on whether the user is stationary (e.g. at adesk), or travelling (e.g. in a car, bus or train). For example,learning experience engine 110 may refrain from sending tests to a userduring periods in which the learning space platform 120 is providinginformation that indicates that the user is travelling.

As additional examples, a particular type of mobile computing device maynot possess the ability to install a particular tool, such as a flashplug-in. Further, the device may currently be low on battery power,making a learning mode that requires less screen use more desirable, oreven the only option. The size of the computing device's screen may alsobe considered when making a content recommendation. For example, acollaborative tool, such as a chat session or shared whiteboard system,may require a larger screen to be effective, and thus may not beappropriate for a mobile computing environment.

The tool (e.g. a cognitive tutor) selected to deliver content to theuser may or may not be aware of the student's cognitive DNA—it is theresponsibility of the learning space platform 120 to launch the toolwith the appropriate configurations for customization supported by thetool. Advanced tools or newly created tools on the platform may chooseto use the information on the cognitive DNA in order to personalize theexperience.

The Learning Content Manager

Learning content manager 130 stores and manages learning content.Learning content includes any content that may be used to learn a skillExamples of learning content include text, audio files such as mp3files, video files such as QuickTime files, interactive flash movies, orany other type of multimedia content.

In an embodiment, learning content manager 130 includes a contentrepository and a content categorization system for storing andorganizing learning content. The content repository stores content innon-volatile memory, such as a shared hard disk system. The contentcategorization system provides content indexing services, along with aninterface for creating and associating metadata with content stored inthe content repository.

Content may be associated with metadata that describes the content. Thismetadata assists course developers in determining which content may beappropriate for learning particular skills. For example, metadataassociated with a video may include a title attribute that includes thetext “how to factor polynomials.” Other attributes may include a generalcategory, such as “math” and a content type, such as “QuickTime video.Metadata may be embedded within the content being described by themetadata, may be in a separate Meta file such as an XML file that isassociated with the content being described, or may be stored in adatabase with an association to the content being described.

Learning content manager 130 also includes content delivery logicconfigured to manage requests for content that is stored in the contentrepository. For example, some content may be streamed in order topreserve bandwidth. In some cases, it makes sense to deliver allrequired content for a particular course at the same time, such as whenthe student expects to be without Internet access for an extended periodof time. Thus, learning content manager 130 may be directed by learningexperience engine 110 to deliver content in a particular way, dependingon attributes of the student. In addition, certain content formats maynot be supported by certain devices. For example, content delivery logicmay choose or even change the format of the content being delivered ifthe device requesting the content does not support a particular format,such as the flash format.

The types of content and tools that may be used with learning managementplatform 100 are not limited to those discussed herein. Instead, theexamples provided are meant to serve as possible types of content andtools that may be used, and are non-limiting examples.

The Learning Tools Manager

Learning tools represent software required for delivery of learningcontent. Learning tools may include, for example, video players, virtualwhiteboard systems, video chat software, and web browsers. A web browserplug-in may also be a learning tool. Each of these tools may be requiredin order for the student to view the recommended content. For example, arecommended piece of content may consist of a flash movie. A flashmovie, in order to be played, requires a flash player to be invoked bythe learning space platform 120 running on the student's clientcomputing system. Another example of a learning tool may be a gamesystem.

Learning tools manager 140 manages and organizes learning tools. In anembodiment, learning tools manager 140 includes a tool informationdatabase, a tool repository, tool selection logic, and tool deliverylogic.

The tool information database includes information about each learningtool, such as whether or not the tool will work with a particular typeof client, such as a handheld device. For example, a flash player maynot work on some mobile devices. Other information in the toolinformation database may include information, such as network locationinformation, that enables learning space platform 120 invoke thedownload of a tool that is not stored in the tool repository. Forexample, a URL of a required tool, which may not be stored in therepository, may be provided to a student, along with a prompt todownload the tool.

The tool repository provides storage for downloadable tools. In anembodiment, tool selection logic may assist learning space platform 120in selecting a tool that is appropriate for a particular client device.For example, tool selection logic may determine that a particular mediaplayer, such as a video player or browser plugin, is required in orderto view content that has been suggested to the student. Tool selectionlogic may determine that the student is using an Apple Macintoshcomputer, and provide the version of the tool that runs on Applemachines for download. In addition, tool selection logic may determinethat no tool that plays the suggested media is available for theplatform. Tool selection logic may then report this to learningexperience engine 110, which will make a new content recommendation inan embodiment.

Tool delivery logic is configured to manage requests for tools that arestored in the tool repository. For example, a student may require a toolthat takes a significant time to download. Tool delivery logic may breakup the tool into smaller parts for separate download in order to ensuresuccessful delivery of the tool in the case of a lost connection. Inaddition, tool delivery logic may interact with a download manager inthe learning space platform 120.

The Data Analysis Engine

Data analysis engine 150 performs a detailed analysis all informationgathered by other elements of the learning management platform 100 inorder to identify correlations between student attributes and learningexperiences. For example, changes in user profile information,assessment results, user behavior patterns, clickstream data, learningevolution information, resource monitoring information, or any othertype of information available may be analyzed by data analysis engine150. Data need not be structured in a particular way to be analyzed, andmultiple sources of data may be analyzed in real time. In addition,multiple data sources may be aggregated, even if each source providesdata in a different format or structure. The aggregated data may then befiltered to provide a detailed cross platform analysis on specific datarelationships.

Data analysis engine 150 may analyze profile information to determinegroups of users that are similar to one another. In addition, dataanalysis engine 150 may determine the times of day, locations, and othertransient attributes that are associated with a high degree of successfor a student. For example, based on results of assessments taken atdifferent times of the day, data analysis engine 150 may determine thata particular student studies more effectively between the 9 am and 11 amin the morning, and that the positive effect is magnified when thestudent studies at a particular bookstore in town. Any attribute may bestudied by data analysis engine 150 to determine correlations betweenstudent attributes and learning effectiveness.

In an embodiment, data analysis engine 150 operates in a clusteredcomputing environment, using existing software, such as Hadoop by theApache Hadoop project. In other embodiments, custom implementations ofHadoop or other software may be used, or a completely custom dataanalysis system may be used. Data analysis engine 150 includes reportinglogic configured to provide detailed reports to learning experienceengine 110. These reports assist learning experience engine 110 inmaking learning recommendations.

The Knowledge Base

Knowledge Base 160 manages persistent data and persistently storessnapshots of certain transient data. For example, student categorizationinformation, student study group information, cognitive DNA relationshipinformation, and persistent student profile information may all bestored in knowledge base 160. Although this data is persistently stored,the data may change as required by other elements of the learningmanagement platform 100. For example, data analysis engine 150 mayprovide a report to learning experience engine 110 that causes learningexperience engine 110 to indicate to knowledge base 160, based on thereport, that student categorization information for a particular studentshould be changed. Knowledge base 160 will then alter the persistentdata to reflect the indicated change. In an embodiment, knowledge base160 includes a relational database management system to facilitate thestorage and retrieval of data. Knowledge base 160 is communicativelycoupled to learning experience engine 110, and provides learningexperience engine 110 with student information to assist in creating anindividualized learning recommendation.

The Personal Cognitive DNA Manager

Personal cognitive DNA manager 170 manages data associated withstudents. A collection of data associated with a student is known aspersonal cognitive DNA (PDNA). The portions of a PDNA that are stored atthe personal cognitive DNA manager 170 may be transient data, whilepersistent portions of the PDNA may be stored in knowledge base 160.PDNA data stored in personal cognitive DNA manager 170 may also includereferences to persistent data stored in knowledge base 160. Personalcognitive DNA manager 170 may include a database management system, andmay manage PDNA for all students. In an embodiment, instances ofpersonal cognitive DNA manager 170 may reside on the client computingdevices of students, and may be part of the learning space platform 120.In this embodiment, PDNA for users of the client computing device or theassociated learning space platform 120 may be stored in volatile ornon-volatile memory. A combination of these embodiments may also beused, where a portion of the personal cognitive DNA manager 170 resideson a client while another portion resides on one or more servers.Personal cognitive DNA manager 170 is communicatively coupled tolearning experience engine 110, and provides learning experience engine110 with transitory student information to assist in creating anindividualized learning recommendation. For example, a user's location,local time, client device type, or client operating system may beprovided to learning experience engine 110 to assist in determining whattype of content is appropriate for the environment and device. In anembodiment, personal cognitive DNA manager 170 and knowledge base 160may be combined.

Learning Objects

FIG. 3 is a block diagram illustrating a learning object in anembodiment. A learning object such as learning object 300 is associatedwith data that describes that learning object. The associated data maybe stored in a learning object data structure. In an embodiment, alearning object is referenced by a learning object identifier, andassociated data or references to the associated data may be stored in arelational database, and may reference the identifier to indicate thatthe data is associated with the learning object represented by theidentifier.

Each learning object includes a skill, such as skill 310. A skillrepresents an ability that a student is meant to acquire. For example, askill may represent the ability to perform addition of single-digitnumbers, form a complete sentence using a particular language, or type acertain number of words-per-minute. There is no limit to the complexityor simplicity of skills that may be included in a learning object.

Each learning object includes content, such as content 320. As shall bedescribed in greater detail hereafter, the content of a learning objectmay include, by way of example and not by way of limitation,assessments, remediation data, skills hierarchy data, bloom level data,learning object metadata, and object-specific personalized data.

Content is said to be “included” as part of a learning object, eventhough the content may only be referenced by the learning object, butmay not actually be stored within a learning object data structure.Content may be stored in a content repository and managed by learningcontent manager 130. In an embodiment, content is “tagged” with metadatadescribing the content, such as keywords, skills, associated learningobjects, the types of learners (e.g. visual) that may benefit from thecontent, the type of content (e.g. video or text), and statisticalinformation regarding the content usage. Learning space platform 120 andlearning experience engine 110 may be authorized to add, remove, oralter tags associated with content via the learning content manager 130.

Skills Hierarchy

Recommendations generated by learning experience engine 110 are based inpart on a skills hierarchy having learning objects in an embodiment.FIG. 2 is a block diagram illustrating a skills hierarchy structure thatmay be used in an embodiment. Skills hierarchy 200A includes learningobjects 201-208, 211, and 214. Skills hierarchy 200B includes learningobjects 207-213 and 215-217. A skills hierarchy may represent a group ofskills, a portion of a course, a course, a field of study, a certificateprogram, a degree program, an individual competency map that representsthe skills acquired by a student, or any other education relatedstructure. Skills hierarchies may be mapped to a wide variety of variouslearning theories, content types, and modes.

Links between objects in the hierarchy represent the relationshipbetween those objects. For example, a link between two learning objectsmay mean that the subject matter covered in one of the learning objectsbuilds on the subject matter covered in the other learning object. Adifferent link may indicate that one learning object is a prerequisiteof the other.

Certain skills may be required in order to begin learning more advancedskills. According to skills hierarchy 200A for example, the skillassociated with learning object 205 must be acquired before advancing tothe skill associated with learning object 208. In this way, the learningskills hierarchy is hierarchical. However, a non-hierarchical approachmay be used in an embodiment. For example, a non-hierarchicaldirected-graph approach may be used in an embodiment that is based on adifferent learning model.

The Skills Hierarchy Manager

Skills hierarchy manager 180 manages skills hierarchy information thatdescribes the relationships between skills. For example, a student maybe required to learn how to add and subtract before he learns how tomultiply and divide. A complete skills hierarchy may be made up ofsmaller interconnected skills hierarchies that represent smaller groupsof skills, which may represent all of the skill required to complete atraditional educational course or degree program. The nodes in thehierarchy correspond to learning objects. Since a single node may beconsidered a prerequisite for many other nodes, and many nodes may beprerequisites for a single node, the hierarchy may be multidimensional.

Skills hierarchy manager 180 manages the relationships between learningobjects. For example, a relational database may be used to keep track ofthe node dependency information. Skills hierarchy manager 180 alsostores object-specific data that describes skills, remediationinformation, assessment information, skills hierarchy associationinformation, learning theory information, content information, toolsinformation, and other metadata associated with learning objects. Skillshierarchy manager 180 interfaces with learning experience engine 110 andprovides learning experience engine 110 with skills hierarchy data toassist learning experience engine 110 in creating an individualizedlearning recommendation. The skills hierarchy information provided tolearning experience engine 110 may include whole or partial skillshierarchies, and object-specific data may be filtered according toparameters passed to the skills hierarchy manager 180 in a request fromlearning experience engine 110.

Learning Theories, Models, and Strategies

Learning theories address how people learn by providing a paradigmthrough which to view a learning objective. Although theories areabundant, three well-understood theories are behaviorism,constructivism, and cognitivism. Behaviorism is a view that is based onthe assumption that people learn based on stimulation. Positive ornegative reinforcement shapes the behavior of the student.Constructivism is a view that assumes that the student learns as anexperience, and that the learner actually constructs information.Cognitivism is a view that assumes that people process information, anddo not merely react to stimulation.

Learning models are created based on learning theories, and are meant tooperationalize one or more learning theories. Different instructionalstrategies may be used, depending on the model to be implemented.Techniques include lecturing, case study, collaboration, one-on-one,direct instruction, and indirect instruction. Content and tools thatfacilitate the delivery of content can be used to implementinstructional strategies. Examples of content include text, video,audio, and games. Examples of tools include video players, browserplug-ins, e-book readers, shared whiteboard systems, and chat systems.Many other examples of content and tools exist.

The theories and models together have been created to classify, adaptand guide the process of educating a student, taking into considerationthe characteristics of the student that are pertinent for learning. Todate, learning models have been implemented in application “silos” whereeach application implemented a specific model (e.g. intelligent tutors),catering to very specific topics that suited the model's flat hierarchybased algorithms, typically for logical subjects such as mathematics.These types of applications have limited use in broad-based education,such as a 4-year degree program or inter-disciplinary subjects such asEconomics, and Marketing.

The Learning Modules

Learning modules can be optionally integrated into learning managementplatform 100, in an embodiment. Learning modules generally representthird-party applications or stand-alone applications withdecision-making capabilities. In an embodiment, learning experienceengine 110 may request input from a learning module such as learningmodule 190A, and then take the input into consideration in making theoverall learning recommendation decision. For example, learning module190A may recommend that a first content item be delivered to a student.Learning experience engine 110 may then determine that the first contentitem should not be provided to the student for reasons that learningmodule 190A is unable to consider, so learning experience engine 110 mayrecommend a similar content item, such as an item in a text formatrather than an audio format.

Learning system modules 190A, 190B, and 190C provide additionalfunctionality to learning management platform 100, and may be based onmodels other than those described herein. Application ProgrammingInterface (API) 115 provides a programming interface for learning systemmodules such as learning system modules 190A, 190B, and 190C. Learningsystem modules 190A, 190B, and 190C are communicatively coupled tolearning experience engine 110 via API 115, and also may becommunicatively coupled to other elements of learning managementplatform 100, such as learning content manager 130. Learning systemmodules 190A, 190B, and 190C may include third party or custom systems,hardware, or software modules.

Personal Cognitive DNA

In an embodiment, learning models and theories may be operationalizedusing each student's PDNA to create an individualized learningexperience for students. Although PDNA is referred to herein as“personal cognitive DNA,” this label does not indicate that PDNA datacollection and usage is limited to embodiments that are based uponcognitive models. PDNA data includes information about a student'scognitive strengths and weaknesses (as well as preferences) that areprovided explicitly by the student or inferred by the system as thestudent interacts with the system and the outcomes are measured.

PDNA may be used in any embodiment, independent of any particularlearning model. PDNA data is a collection of data associated with astudent. Transient profile data may be stored in the personal cognitiveDNA manager 170, while persistent profile data may be stored inknowledge base 160. PDNA data stored in personal cognitive DNA manager170 may include references to persistent data directly or indirectlyassociated with the student that is stored in knowledge base 160.

In order to provide the right experiences to the student, in addition tothe proper tools and models, the system must collect and maintain adynamically updating rich data layer to support predictive educationmodels. A rich data layer generally refers to information that isgathered and linked to create intelligence that may be used to informlearning experience engine 110, which uses this information to generatelearning recommendations. In one embodiment, the rich data layer isdynamically updating in that the data being collected changes over time,and data that does not conform to the changes becomes incorrect. Forexample, as a student achieves a high degree of proficiency with aparticular skill, data that suggests that the student needs to becomeproficient with the skill becomes outdated and incorrect. Thus, thedynamic data layer must keep up with the current information availablefor each student.

Each student using learning management platform 100 is associated withPDNA for that student. When a student is new to the system, the PDNA forthat student may contain minimal information, such as demographicinformation, a student's declared major, self-proclaimed learning stylepreferences, and imported transcript data such as grades and courseworkdone at other institutions. However, as the student begins using thelearning management platform 100, vast amounts of data may be collectedand analyzed by data analysis engine 150, resulting in new PDNAinformation that describes how the student learns, what level thestudent has achieved in a particular course, whether the studentunderstands a particular concept or possesses a particular skill, thepace at which the student learns, or even the time of day the student ismost likely to correctly answer a question.

Many of these student attributes change over time, but may still beconsidered persistent based on the frequency of change. For example, astudent may initially be a visual learner, but may later learn moreefficiently by reading text-based material. Other student attributes maybe more transient in nature. For example, the PDNA may include data thatidentifies the student's current location, what client computing devicethey are using (e.g. iPhone, laptop, or netbook), what operating systemthey are using, whether or not their web browser supports the Flashplug-in, or whether the student sets his status as “tired.”

It may be the case that a particular student performs differentlydepending on environmental factors, while another student may be capableof learning regardless of the environment. For example, one student maybe able to study on a commuter train while another may not. One studentmay be capable of learning via an audio program while another requirestext information or video. In addition, certain tasks may be reservedfor particular times or places in order to calibrate the system. Forexample, the system may be configured to only offer assessments when astudent's transient PDNA data shows that the time in the student'scurrent time zone is between certain daylight hours.

Metrics may be assigned to particular attributes in each student's PDNA.For example, metrics may describe expected or historical success withdifferent learning styles. These metrics may help learning experienceengine 110 determine whether the student is successful whenparticipating in collaborative learning exercises, or whether thestudent would benefit more from self-study. A student may have a metricof “7” for the attribute “visual learner” and a metric of “2” for the“audio learner” attribute. Higher scores are not necessarily the onlyfactor used in determining the learning strategy for the student,however. For example, the learning track that the student is on mayactually require that the student develop listening skills. Therefore,the learning experience engine 110 may recommend a learning experiencefor the student that will bolster his ability to learn via audio. Inother words, the system will teach the student the underlying skillsrequired to allow the student to become a successful audio learner.

PDNA for a particular student may be analyzed and compared to PDNA ofother students to make learning experience predictions. For example,data analysis engine 150 may analyze the PDNA information for allstudents in the system, over time, to predict various things, including:what method of learning is best for each student, which track will yieldthe highest chance of success for a given student in a particularprogram, which programs the student would be successful in, whichcourses the student may be expected to struggle with, and even whichcareer would best suit the student.

Snapshots and Learning Intervals

As students use learning management platform 100 to manage theirlearning needs, natural checkpoints emerge from the data. The frequencyof these checkpoints can be adjusted by system administrators, but aregenerally related to a level of achievement reached by the student forwhich the checkpoint was generated. For example, a checkpoint may begenerated for a student each time that student successfully demonstratesa skill associated with a learning object.

In an embodiment, snapshots of personal cognitive DNA information aretaken at each checkpoint. A snapshot is a static record of PDNA as itexisted at the time of the checkpoint. Each snapshot is stored in atemporal database or other data storage mechanism, such as knowledgebase 160.

Since a new snapshot can be taken for each learning interval (the timebetween checkpoints), the PDNA information stored in the snapshot may beused by learning experience engine 110 to formulate recommendations forother students that may have personal cognitive DNA that is similar tothe snapshot.

DNA Fingerprints

An analysis of hundreds, thousands, or even larger numbers PDNA datasets yields statistically valid cognitive DNA “fingerprints”, in anembodiment. A DNA fingerprint is based on aggregate PDNA data, which mayinclude PDNA snapshots. Generally, a DNA fingerprint is made byselecting a set of PDNA data having one or more PDNA attributes incommon and generating a single profile that is representative of theentire set.

For example, a DNA fingerprint may be generated for students that haverecently completed a learning object that teaches the calculus skill oftaking the derivative of a second degree polynomial. The recency of thecompletion of the learning object is determined based on the time thatthe data was stored, so snapshots that were taken at checkpointsoccurring immediately after students completed the learning object willqualify for inclusion in the set of PDNA data considered for use in thecreation of the DNA fingerprint. Each attribute in the PDNA dataconsidered in the creation of the DNA fingerprint may be aggregated,averaged, or otherwise considered, resulting in a fingerprint of thatattribute. For example, if the average value of the attribute “abstractlearning ability” is “80” in the PDNA data in the set, then the DNAfingerprint may inherit this value for the same attribute. Any method ofconsidering or combining PDNA data to generate DNA fingerprint data maybe used. For example, the lowest value, the median value, or a sum ofthe values may be used as the fingerprint value for a particularattribute. Some attributes, especially those with very littlecorrelation to the common PDNA attribute, may not be assigned a DNAfingerprint value, or may be assigned a NULL value, indicating thatconclusions about that attribute are statistically invalid for that setof PDNA. Once each attribute has been considered for the set, then theresulting values for each attribute are stored in one or more records asa DNA fingerprint for that set of PDNA data.

Using Snapshots and DNA Fingerprints to Make Learning Recommendations

A database of snapshots and DNA fingerprints may be used to shortenlearning experience engine 110 recommendation decision making time bymatching a student's PDNA with a pre-existing DNA fingerprint.

Because fingerprints are based on snapshots, some PDNA data used togenerate the matching fingerprint may have already advanced far beyondthe stage when the snapshot was taken. For example, a snapshot that wastaken two years ago may be used in the creation of a DNA fingerprint.The attributes of the PDNA that change over time, such as those that arebased on location or “last learning object completed,” may be reflectedin the fingerprint, but not in the current PDNA of the student orstudents whose data was used to create the fingerprint. Because of this,a student whose current PDNA has a high correlation with a fingerprintthat is based on an old PDNA snapshot of another student may beconsidered similar to an “old” version of that other student.

If fingerprint PDNA data indicates that students having a particularattribute progressed at a particular rate or excelled in a particularsubject, it may be assumed that students with a matching PDNA willexperience the same success. As time goes on, students associated withthe PDNA data upon which a fingerprint is based may become less and lessalike, and the diversion of some attributes of the fingerprint mayresult in less valid prediction information if a fingerprint were to bebased on the same set of students, but using snapshots associated withsubsequent checkpoints. However, recommendations that are based on acomparison between a student's current PDNA and fingerprints based onsets of PDNA data having the same values for attributes as the student'scurrent PDNA will be highly accurate. The student's PDNA may be matchedto a new fingerprint whenever necessary, desired, or requested.

Learning recommendations generated by learning experience engine 110 arenot limited to suggesting which course a student should take next. Astudent's PDNA may be used to make very specific recommendations basedon a determination that the student is cognitively similar to one ormore other students. The fingerprint method described above may be used,matching the student based on student grouping criteria that is storedin knowledge base 160. In addition, any other matching technique may beused.

An individualized learning experience for each student may be achievedby providing real-time recommendations based on predictive informationassociated with cognitively similar students. When a first student is nolonger cognitively similar to a second student, the second student'sprofile is no longer considered when selecting which learningrecommendations to give to the first student. Thus, while a shortportion of the learning experience, such as learning a particularconcept or skill required for a portion of a course, may be similar tothe experience of others, students may complete an overall learningprogram of study by taking a completely unique path with respect toother students.

Temporal Fingerprint Paths

A temporal fingerprint path generally refers to a series of DNAfingerprints that are generated based on the same set of students. Forexample, consider a situation in which a PDNA snapshot for Lenny Learnerthat was taken one year ago was used in the fingerprint creation processof a particular fingerprint. All PDNA sets used in the particularfingerprint, including Lenny's, may then be analyzed for subsequentsnapshot information to create a temporal fingerprint path.

In an embodiment, each fingerprint that is part of a temporalfingerprint path may be based on snapshots that are not associated withthe same “time.” Instead, each fingerprint may be based on a series ofsnapshots for each user in the set of users used to generate thefingerprint without respect to the time the snapshots were taken. Forexample, Lenny Learner and Laura Learner may each be associated withPDNA information used in generating a temporal fingerprint path thatincludes five fingerprints. The snapshots associated with Lenny that areused to generate the five fingerprints may have been generated over aperiod of five years, starting seven years ago. The snapshots associatedwith Laura that are used to generate the five fingerprints may have beengenerated over a period of two years, starting three years ago. In thisembodiment, the temporal fingerprint path may be designed to representprogression through a series of learning objects, with Lenny and Laurabeing chosen based on their involvement with the same learning objects.

In another embodiment, a temporal fingerprint path may be more strictlybased on time. For example, the snapshots associated with Lenny that areused to generate the five fingerprints may have been generated over aperiod of exactly two years, starting seven years ago, and the snapshotsassociated with Laura may have been generated over a period of exactlytwo years, starting three years ago. Thus, the length of time from thebeginning of the fingerprint path is given added importance.

However, snapshots for a particular student occurring earlier thansnapshots for that student that are used in generating a firstfingerprint are unlikely to be used in the generation of a secondfingerprint in a temporal fingerprint path. A temporal fingerprint pathgenerally implies progression, so a subsequent snapshot for that userwould likely be selected to use in generation of subsequentfingerprints.

Temporal fingerprints are particularly useful in generating long-termlearning strategies for students that are enrolled in a degree orcertification program that has specific requirements. This isparticularly true for students for which very little PDNA informationhas been gathered. For example, a student may have no history with thesystem, but may be enrolled in a Computer Science degree program. Atemporal fingerprint path may be generated based on the set of studentsthat have completed the Computer Science degree program. Based on thetemporal fingerprint path, learning experience engine may provide anexpected long-term learning strategy to the student. As more studentsprogress through the Computer Science degree program, the set ofstudents considered for generating a temporal fingerprint path for theprogram may change, altering the temporal fingerprint path. For example,the set of students considered for generating a temporal fingerprintpath may be based only on students that have completed the ComputerScience degree program within the last five years in an embodiment.

Other factors may also be considered when selecting the set of studentsthat are used to generate a temporal fingerprint path. For example,degree programs change over time, and today's students may need toacquire different skills than yesterday's students to complete theComputer Science degree program. Thus, in another embodiment, onlystudents in the same “version” of the Computer Science degree programare considered when creating a temporal fingerprint path for aparticular student.

Content Feedback Interface

A content feedback interface may be provided with learning spaceplatform 120. FIG. 4 illustrates a browser window 400 with a videocontent window 410 and a content feedback interface 420 in anembodiment. By selecting a number of “stars,” a student may indicate heroverall satisfaction with the content displayed in video content window410. The content feedback interface allows students to provide feedbackabout the content, and whether or not the content or the tool thatenables the content was appropriate for the situation in which it wasused.

In addition to the features described with respect to FIG. 4, a contentfeedback interface may allow users to rate content based on manyfactors, such as whether they found the content convenient, easy tounderstand, appropriate as a next step to the previous content used, orwhether the student likes the content. Text entry fields and otherinterface elements may be used as appropriate for gathering additionalfeedback data from students.

All of the content feedback may be stored in learning content manager130, and used to tag content. This allows users to force contentadaptation and alter the learning path of other students by increasingor decreasing the likelihood that the content will be used in aparticular situation or for a student with a particular type of PDNA.

For example, two different video presentations may be used to teach thesame skill. Although the first video presentation may have a highersuccess rate than the second video presentation, the second videopresentation is preferred by students. Taking student preferences intoconsideration, learning experience engine may recommend the second videopresentation when the disparity between the preferences associatedbetween the two presentations reaches a certain threshold. That is tosay, when the less effective video presentation is much more preferredby the students than the first video presentation, then the second videopresentation will become the default presentation recommended tostudents.

Assessments

In one embodiment, assessments, such as assessments 340 are included ineach learning object. A student uses assessments to demonstrate theskill associated with the learning object. More than one assessment maybe included in a learning object. Learners may be required tosuccessfully complete all, or a subset, of the assessments in order toreceive an advancement recommendation from learning experience engine110. The number and type of assessments required may depend onhistorical data describing past experiences with the learning object orother related learning objects. For example, if a student is known tohave difficulty with learning objects that include mathematical skills,a student may be required to successfully complete relatively moreassessments for a particular learning object associated with a mathskill.

Remediation

Remediation data, such as remediation data 330, may also be included ina learning object. In an embodiment, remediation data includes detailedinformation describing which learning objects are preferred remediators(objects that assist in skill building) for the current learning object,or even which learning objects the current learning object is apreferred remediator for. For example, a student may experience littleor no success in completing the assessments associated with learningobject 212. Learning object 212 may include remediation informationlisting learning objects 205 and 209 as good remediators. Remediationmetrics may be used to help learning experience 110 engine determinewhich remediator to recommend. If learning object 205 has a higherremediation metric than learning object 209, then learning object 205may be selected, and the student will be provided content andassessments associated with learning object 205. When the student isready, he will advance to learning object 208, and then make anotherattempt at learning object 212.

Remediation information and remediation metrics may be different fordifferent types of users, or users having particular types of PDNA. Forexample, learning object 205 may have a higher remediation metric thanlearning object 209 for group A, but learning object 205 may have alower remediation metric than learning object 209 for group B. Inaddition, remediation information is not limited to learning objectrelationships. Remedial relationships may be formed at the assessmentlevel, even if assessments are not within the same learning object orpresent skills hierarchy.

Skills Hierarchy Data

Skills hierarchy data such as skills hierarchy data 360 describes alearning object's skills hierarchy associations. Each learning objectmay be included in multiple skills hierarchies. For example, learningobject 207 is included in skills hierarchy 200A and skills hierarchy200B. Each of skills hierarchy 200A and skills hierarchy 200B representa skill set. For example, skills hierarchy 200A and skills hierarchy200B may each represent a course, such as English Composition or LinearAlgebra. Overlap in skills hierarchy data illustrates themultidimensional nature of the larger skills hierarchy used in learningmanagement platform 100.

Bloom Level Data

Bloom level data such as bloom level data 370 may be included in alearning object. Bloom level data identifies bloom taxonomy informationfor one or more courses in which the learning object is required. Thesix bloom levels describe whether a user has knowledge and can remembera concept, understands and can describe and explain a concept, can applythe concept, can analyze based on or according to the concept, canevaluate based on or according to the concept, or can create based on oraccording to the concept. A particular learning object may be associatedwith one bloom level for a particular skills hierarchy, but may beassociated with a different bloom level for another skills hierarchy.For example, learning object 211 may be at the “evaluate” bloom levelfor skills hierarchy 200A, but may be at the “apply” bloom level forskills hierarchy 200B.

Learning Object Metadata

Metadata such as metadata 350 may include data about the learningobject. For example, version information, change tracking information,or other information about the learning object may be stored in metadata350. Metadata 350 may also include data that should be communicated tolearning space platform 120 when learning space platform 120instantiates the learning object and associated content and tools. Forexample, a description or mini-syllabus describing the learning objectand associated skill may be sent by learning experience engine 110 whenthe learning object is selected by learning space platform 120.Additional metadata may be stored in learning object 300. Notes, futureimplementation specifics, and developmental skills hierarchy identifiersare all examples of data that may be stored in metadata 350. The amountof metadata that may be stored about an object is bound only to thesystem constraints or administrator discretion. Thus, metadata 350 maybe considered a catch-all for other data that is to be associated with alearning object. In addition, learning objects are extendable, and mayhave other data categories associated with them that are not describedherein.

Object-Specific Personalized Data

Detailed personalized information may be stored on a per-person,per-object basis. For example, a particular student may have reached acertain level of understanding with respect to a particular skill thatis associated with a particular object. The information stored may bemapped to Bloom taxonomy levels, and may include additional studentmetadata that describes the particular experience that the student hashad with the object. For example, metadata may describe how fast thestudent learned the skill, whether the student enjoyed the content usedto learn the skill, and trouble-areas for the student. This informationmay be gathered through frequent assessments, learning activities,learning games, homework assignments, and participation in groupactivities, in addition to other information-generating eventsassociated with the student's interaction with learning managementsystem 100.

Traversing the Skills Hierarchy

The phrase “traversing the skills hierarchy” refers to a student'sprogression through the learning objects in the skills hierarchy. Askills hierarchy for a particular course syllabus, such as skillshierarchy 200B, may include multiple learning objects, such as learningobjects 207-213 and 215-217. The skills hierarchy may be for astatistics course, and require all of the skills included in theassociated learning objects to be acquired by the student in order tomark the course as “completed” for the student. Skills hierarchy 200B isnot necessarily representative of a desired or anticipated size of askills hierarchy for a particular course. For example, a skillshierarchy for a statistics course may include hundreds of learningobjects, each directed to a granular skill or concept.

A student begins traversing the skills hierarchy by receiving a learningrecommendation from learning experience engine 110. If the student hasnot interacted with learning management platform 100 before, then thestudent will need to provide information to help build a PDNA. Forexample, the student may need to take one or more pre-assessments todetermine the skill level that the student has with respect to hisprogram. Generally, the more pre-assessments the student takes, the moreaccurate the initial recommendation will be. Other useful informationsuch as transcript information from higher education institutions mayalso help to build a PDNA for the user.

Learning experience engine 110 takes into account information stored inthe student's PDNA such as pre-assessment information, informationstored in knowledge base 160, and information learned from data analysisengine to determine which skills hierarchy the student should traverse,and which learning object on that skills hierarchy the student shouldbegin with. For example, knowledge base 160 may include informationabout the student that indicates the student has completed the courseassociated with skills hierarchy 200A, and has therefore completedlearning objects 207, 208, and 211. The learning recommendation maytherefore not consider these learning objects as required because theyhave been completed in a previous course. Learning experience engine 110may therefore recommend that the student begin at learning object 209 onskills hierarchy 200B.

Learning object 212 may represent the ability to calculate a confidenceinterval, which is used to indicate the reliability of statisticalestimates. Because confidence intervals are typically expressed as apercentage, the ability to calculate percentages may be considered aprerequisite skill, identified by learning object 208. Skills hierarchy200B indicates that learning object 209 also represents a prerequisiteskill for learning object 212. Given this information, and the knowledgethat the student has completed the course represented by skillshierarchy 200A, it may be determined that learning object 209 is theonly prerequisite required in order for the student to attempt learningobject 212, even though learning object 208 was completed as part of aseparate course.

In an embodiment, when a student attempts to complete a learning object,but experiences little or no success completing assessments associatedwith that learning object, the student will traverse the skillshierarchy toward a remedial learning object. For example, if a studentcompletes the assessments at learning object 212 with a small metric ofsuccess, such as answering only 20% of questions correctly or completingtasks incorrectly, learning experience engine 110 may recommend that thestudent move to learning object 203. Alternatively, if a studentcompletes the most difficult or complicated assessments associated witha learning object with complete accuracy, it is possible that the systemis unaware of skills possessed by the student. Learning experienceengine 110 may therefore recommend that the student begin at a muchhigher level, even skipping levels in the skills hierarchy. For example,a student that successfully completes the most difficult assessmentsassociated with learning object 211 with perfect accuracy may receive arecommendation to attempt assessments associated with learning object217 in order to complete the course. Based on the assessmentinformation, learning experience engine 110 may indicate that thestudent has finished the course, or may recommend additional contentthat is appropriate for the student.

It is not necessary that the types of tools or content used forremediation be similar to the tools and content used for the skill thatrequired remedial training. Furthermore, it is not the case that thecourse that includes the skill requiring remedial training be in asimilar category to the learning object required for remediation. Forexample, a group of business students in an MBA business developmentcourse may be working on a learning object using a collaborative toolthat includes a shared whiteboard, a chat session, and a voice overInternet protocol connection. During the collaborative session, however,the students may be required to calculate the net present value (NPV) ofa company, which is the total present value of a time series of cashflows. Cash flows must be discounted to a present value, and then summedtogether. If a particular student has difficulty making the calculation,the present learning object for that student may be interrupted bylearning space platform 120 and retrieve a recommendation from learningexperience engine 110. Learning experience engine 110 may recommend thatthe student use a particular behaviorism-based tool associated withanother learning object in order to acquire the necessary skill tocalculate NPV. Once the student successfully completes the mathematicsassessment, he will then be returned to the collaborative session withthe other MBA students. This example illustrates the platform's unifyingdata concepts and date driven behavior—switching between appropriatelearning models (e.g. a student may have been in a social constructivistlearning space working on a case study on finance when the need forremediation of a concept surfaced. The next experience chosen by theplatform might be an implementation of a cognitive tutor designed formath instruction.

In the previous example, learning experience engine 110 effectivelycreated a mini-course for the student to ensure he has the skillsrequired to be successful on his current track. This is possible partlybecause the learning management platform 100 keeps track of data aboutthe students across different courses, and uses this information to helpthe student traverse the tree. For example, the MBA student may havealready taken algebra, so a short remedial path to remind the studentmay have been the only information that the student needed in order tomove on. However, if the student's algebra course was over two yearsago, and he has taken no math since then, learning experience engine 110may recommend a more detailed review of algebraic concepts, even takinginto consideration future courses that the student is expected to takeand the math skills required for those expected courses.

While the student is traversing the tree, content, tools, and learningobject metadata are changed to reflect the success of the path taken bythe student. For example, a student may perform poorly on one or moreassessments associated with learning object 216. Remediation dataassociated with learning object 216 indicates that learning object 209or a particular assessment or content item associated with learningobject 209 is a good remediator for learning object 216, or even aparticular assessment associated with learning object 216. If thestudent is directed to learning object 209 for remediation, and thenattempts the assessment at learning object 216 again with success, thena remediation metric describing the association between learning object216 and learning object 209 is changed to reflect the success. In thisway, the skills hierarchy itself is dynamic and always changing based oninput from the students.

The distinct paths that a student may take reside in a multidimensionallearning space associated with a skills hierarchy. The path that astudent may take through a multidimensional learning space shouldprovide the most efficient and best outcome for a student. A skillshierarchy includes many learning objects, and learning objects includeor are associated with a skill and variety of content and assessments.Although two students may be said to have acquired the same skills,completing the same course, each student may have taken a differentlearning path.

For example, student A may be a visual learner, and be cognitivelysimilar to a set of students. These students are places into aparticular group, or “tribe,” based on their cognitive similarities. Inthis example the tribe is referred to as tribe A. Student B, who is partof a different tribe, may learn well by reading text, and gain verylittle benefit from visual learning techniques. Based on thisinformation, learning experience engine 110 will provide differentrecommendations to each student for some learning objects. For example,student A may acquire the skill associated with learning object 206 bywatching a video and completing three assessments that are based on aninteractive game tool. Student B, on the other hand, may acquire thesame skill by reading a chapter in an e-book, and completing a singleassessment requiring a writing assignment.

While traversing the skills hierarchy, students may provide feedback by“tagging” content that they are interacting with. For example, a studentmay particularly enjoy an interactive learning game. The student mayselect a user interface element such as a button labeled with the word“fun” to indicate the preference. This information may be used in laterlearning recommendations for the student and other students that arecognitively similar to the student.

The skills hierarchy may also evolve over time. By analyzing dataassociated with a particular learning object, such as data the shows howwell students in different categories perform with respect to thelearning object, learning experience engine 110 may determine that theobject itself is defective or produces a sub-optimal result. Forexample, if a group of students that tend to perform well do not performwell on assessments associated with a particular object, then it ispossible that the content used to teach the skill is mismatched with theassessments associated with the particular learning object. Learningexperience may generate reports that highlight these deficiencies sothat course developers and content developers may change the learningobject, or introduce an alternate, competitive learning object.Competitive learning objects will be described in greater detailhereinafter.

Targeted Interaction

Students, faculty, and other users of the system may interact with oneanother using collaborative tools and social networking featuresassociated with the learning management platform 100. In an embodiment,any student working on any assignment from any location at any time willbe able to click a single button to inquire who else among the facultyor students is working at the same time, in the same or similar contentarea, and then begin to engage in questions or discussion. For example,a student that is working on a particular learning object may wish tocollaborate with other students, faculty, tutors, or other users of thesystem working on the same learning object.

In an embodiment, the student may press a button or otherwise interactwith a user interface element associated with collaboration logic builtinto learning experience engine 110, thereby generating a collaborationrequest to the learning management system. In response to receiving therequest, the learning management system determines which other users ofthe system are available for collaboration with respect to theparticular learning object. The learning management system then returnsa list of available collaborators to the student. Although thisembodiment is directed to collaboration based on learning objects, thisconcept may be used with any other commonality between collaboratingusers. For example, users may collaborate with other users of the systemthat have some association with a particular skill hierarchy.

In an embodiment, users of the system may define preferences associatedwith collaborative learning. For example, students may wish to workcollaboratively with other students, or may wish to only work withfaculty or tutors. Additionally, students may wish to collaborate onlywith other students that have similar cognitive DNA. These preferencesand other preferences based on any other similarities or differencesbetween users may be stored as part of a student's cognitive DNA, andmay be used to determine the makeup of a collaborative group.

Collaborative groups may also be limited to optimal group sizes. In anembodiment, the group sizes of the groups to which users are assignedmay be based on cognitive DNA similarities between the users. Forexample, a particular type of student may perform better in a smallergroup, while another type may perform well with larger groups. Groupsize may also be based on the type of collaborative tool being used. Forexample, a shared whiteboard system may become less effective as morepeople attempt to draw on it. However, a chat room associated with avideo tutorial may allow for a larger number of users, which may befurther based on the activity level of the chat room.

Students may collaborate with other users of the system that are usingthe same learning tools as themselves, or may collaborate with studentsthat are using a different learning tool than they are. For example, acollaboration may involve one student writing on a white board, andanother student typing in a chat room. The input to the white board maybe dynamically translated to text that appears in the chat room, whilethe text in the chat room appears on the white board.

In an embodiment, a user of the system, such as a faculty member orfirst student may provide a learning tool recommendation to a secondstudent based on the information shared in the collaboration session.

Competitive Learning Objects

Each node in a skills hierarchy has been described herein as having asingle learning object. However, nodes in a skills hierarchy need not belimited to having only one associated learning object. Competitivelearning objects that are directed to a similar or identical skill mayreside at the same node in a skills hierarchy. Each learning object mayinclude different content and assessments than the other learningobjects that occupy the node.

Metadata associated with each learning object of a node may indicate thesuccess or failure of that learning object across categories. Forexample, one learning object may be more successful for nursing studentswhile another learning object may be more successful for engineeringstudents. Although different categories of students are discussed, otherfactors may be used to measure the success of a learning object. Forexample, a first learning object may be a better remediator than asecond learning object for a particular node in a skills hierarchy thatis associated with a different course.

Competing learning objects are also useful when introducing new contentand skills into the learning management platform 100. Adding a learningobject as a competitive learning object rather than replacing theexisting learning object allows for a trial period to determine whetherthe new learning object provides an improvement over the old.Integration of new objects and content may therefore be seamless. It mayalso turn out that the best action is to leave all of the competinglearning objects in place because they each provide a different benefitthat the other learning objects sharing the node space cannot.

Learning System Modules

In the illustrated embodiment, the learning management platform 100includes an Application Programming Interface (API) 115 that isconfigured to interact with learning system modules. API 115 may beimplemented over a network connection or any other communication method.Learning system modules may include third-party artificial intelligencesystems or other decision making recommendation, and learning systems.These learning system modules may have access to other elements of thelearning management platform 100, such as personal cognitive DNA manager170, or may perform independent of these other elements. In anembodiment, skills hierarchy manager 180 operates as a learning systemmodule, and interfaces with API 115. Learning modules are not limited bylearning management platform 100, and may include additional supportingsystems, hardware, networking equipment, cloud-computing systems, andexternal data sources. Learning modules may include any software,hardware, or network resources necessary to perform optimally.

Although a skills hierarchy-based system has been presented, such asystem, which may include skills hierarchy manager 180, is not requiredfor learning space platform 100 to function. Learning system modulessuch as learning system modules 190A, 190B, and 190C, may be configuredto operate based on any model or criteria. Rule-based models, forexample, may include a decision-making structure that is much differentthan a directed graph approach, taxonomy, or the skills hierarchydescribed herein. For example, a learning system module configured tooperate using a rule-based model may receive, as input from learningexperience engine 110, a text-based answer from a student. Based onrules within the model used by the learning system module, the modulemay reject the answer, and may provide associated information tolearning experience engine 110. The rule may be based on linguistics orany other criteria. For example, the learning system module may detectthat the text-based answer was misspelled, or that a word did not havethe required number of syllables, or that the sentence or paragraphstructure was incorrect. A learning system module may even include rulesto determine that an essay is missing a thesis sentence.

Learning system modules need not be tied to one particular model. Forexample, a rule-based model may be combined with another model, such asa directed graph-based model in order to achieve the advantages of bothmodels. In addition, no formal model is required in order to interfacewith learning experience engine 110. For example, a subject-specificcognitive tutor may be developed with no regard to learning theorywhatsoever, and use a completely new structure and means for decisionmaking, and that cognitive tutor may function as a learning systemmodule that may be “plugged-in” to learning experience engine 110 byusing an interface compatible with API 115.

Competing learning system modules may be used, allowing learningexperience engine 110 to receive input from multiple modules, and thenprovide the best overall learning experience for the student. Forexample, an artificial intelligence based module, a taxonomy module, anda rule-based module that are all designed to teach a student to play thegame of chess may be concurrently communicatively connected to API 115.Each module may store metadata associated with learning content storedin learning content manager 130, learning tools stored or indexed inlearning tools manager 140 or any other data required to provide aresponse to learning experience engine 110. Learning experience engine110 may request information, data, or recommendations from each modulethat it then uses to provide a learning recommendation to the student.Learning experience engine 110 may determine that the experienceprovided to the student is superior when a particular module is usedduring a particular portion of the learning experience. This decisionmay be based on any criteria. For example, the experience may be deemedsuperior based on the speed at which the student progresses, based onmore superficial information, such as the delay incurred by using a lessefficient module.

Since learning experience engine 110 is capable of concurrentlyreceiving input from more than one module, new modules may be added tothe learning management platform 100 and gradually integrated into thesystem. Each module may increase in importance and influence based onthe merit as determined over time by learning experience engine 110.Alternatively, a new module may be configured by an administrator of thelearning management platform 100 to be the primary module used for aparticular subject, skill, or task. For example, an independently testedand proven module may be integrated into learning management platform100 and immediately promoted as the module with the most merit,overriding any determination made by learning experience engine 110.Learning experience engine 110 may then be configured to performadditional merit determinations for the new module, as well as existingmodules.

Alternative Embodiments

In an embodiment, learning experience engine 110 may provide more thanone recommendation for the student, along with data about eachrecommendation such as data associated with content or learning objects.The student then may be able to choose a learning experience based onpreference. Learning experience engine 110 may gather preferenceinformation for further processing by data analysis engine 150, and usethe analysis to assist learning experience engine 110 in determiningfuture recommendations for that user or cognitively similar users.

In another embodiment, multiple recommendations are provided, butlearning space platform 120 may make the final decision regarding whichcontent, assessment, or learning objects are presented to the user. Thismethod allows local logic to offload processing of the final learningrecommendation based on transitory environmental factors that areoverriding, such as location, mobile network signal strength, orlighting detected by the device.

In another embodiment, learning experience engine 110 may provide agroup of learning objects and associated content, assessments, andtools, along with logic to allow learning space platform to determine,based on assessment performance, which of the group of learning objectsshould be presented next. This method allows for extended offlinelearning. In this embodiment, learning space platform 120 may stilldetect connectivity and request an additional group of learning objectsbased on assessment performance, advancing the state of the locallystored data in order to keep a cache of offline learning information,content, and tools available at all times.

In another embodiment, a control system is included in learningmanagement platform 100. The control system may be configured to selectthe next steps of the learning space navigation automatically or enablethe navigation to include explicit steps set up by a faculty member, ora hybrid operation where a faculty member may decide to selectivelyoverride some of the steps of the skills hierarchy traversal.

In an embodiment, learning management platform 100 includes apersonalized assessment system that is capable of taking different goalsets from students, faculty, institutions, and employers, and selectingthe right set of assessments to ensure that the student has mastered theright skills in-line with the goals.

Example Method for Providing Learning Recommendations Based On ProfileSimilarity

In an embodiment, profiles are maintained for students by personalcognitive DNA manager 170 and knowledge base 160. These profiles mayinclude PDNA, and describe one or more education-related attributesassociated with students. A profile snapshot (PS1T1) that represents thestate of the profile of a first user at a particular point in time (T1)is also maintained. A current profile (PS1T2) that represents a state ofthe profile of the first user at a second point in time (T2) is alsomaintained for the first user. Another current profile (PS2T2) thatrepresents a state of the profile of a second user at a second point intime (T2) is also maintained.

In response to a request, for content, from the second user, learningmanagement platform 100 determines, based at least in part on anattribute of the profile snapshot PS1T1 of the first user that is not anattribute of the current profile PS1T2 of the first user, that asimilarity exists between the current profile of the second user and theprofile snapshot of the first user. Based on the similarity between theout of date profile (PS1T1) of the first user and the current profilePS2T2 of the second user, a content item is selected for delivery to thesecond user.

For example, a first user may currently learn most effectively by usingvideo-related learning tools, but that at time T1 the first user mayhave learned most effectively by reading text. If the second user, whocurrently learns most effectively by reading text, needs a learningtool, the current profile P2T2 of the second user can be matched to asnapshot of the first user's profile P1T1 (that was taken at time T1) todetermine a learning recommendation for the second user. For example, ifat time T1 the first user learned a concept well by reading text X, thenthe learning recommendation may be for the second user to read text X(even though that would not be the learning recommendation that wouldnow be given to the first user).

Example Method for Recommending Content Based on Student Context

In an embodiment, learning objects are maintained by skills hierarchymanager 180. Profiles are maintained for students by personal cognitiveDNA manager 170 and knowledge base 160. Each profile describes one ormore transient attributes that change simultaneously with environmentalor emotional circumstances associated with the corresponding user. Forexample, a student may feel tired or sad, and indicate this through auser interface provided by learning space platform 120. This informationmay be reported to learning experience engine 110, which may store theinformation in knowledge base 160. As another example, learning spaceplatform 120 may be executing on a mobile device with GPS (GlobalPositioning System) capabilities, and may report location information tolearning experience engine 110, which may store the information inknowledge base 160.

A profile may have one or more persistent attributes that describecharacteristics of the student that are pertinent to educationalactivities, such as learning style. For example, a particular studentmay not learn effectively when exclusively using audio content. Althoughtwo students may have the same value for a particular transientattribute, they may have different values for a particular persistentattribute. For example, both students may be on a train (a transientattribute), but one student is unable to learn effectively using thetype of content available for use while in transit (a persistentattribute) while the other student is able to learn effectively usingthe type of content available for use while in transit. Under thesecircumstances, learning experience engine 110 may decide to recommendone learning object to the student who is better able to learn on thetrain. The other student may receive a recommendation to wait until offthe train to continue learning activities.

Example Method for Facilitating Targeted Interaction Between Students

In an embodiment, profiles are maintained for students by personalcognitive DNA manager 170 and knowledge base 160. A student sends arequest for interaction with other users. For example, a particularstudent may want to discuss a particular educational concept with otherstudents. Learning experience engine 110 detects a group of studentsthat are interacting with the same learning object or similar learningobjects to the learning object that the particular student isinteracting with. Learning experience engine 110 then compares theprofiles of the particular user with the group of students that areinteracting with similar learning objects, and determines which of thosestudents are similar to the particular student. At least one otherstudent is selected to interact with the particular student based onthis comparison. For example, a second student may be invited to avirtual whiteboard session or live chat session with the particularstudent.

Example Method for Competitive Learning Objects in a Skills Hierarchy

In an embodiment, a hierarchy of learning objects is maintained byskills hierarchy manager 180. Each learning object in the hierarchy isassociated with a corresponding skill and content items that helpstudents to master the skill. A particular node is occupied by twolearning objects that are competing with one another to be the preferredlearning object at that node. Both learning objects are associated withthe same skill. However, the learning objects may be associated withdifferent content or logic that defines different content preferencesthat cause different content to be delivered to different students, eventhough the circumstances of each student may be the same.

For example, two learning objects residing at the same node in a skillshierarchy may be designed to teach the skills required to performintegration by parts, concept in calculus. The first learning object mayemploy interactive learning games to teach the concept, while the secondlearning object may use a series of videos to teach the concept. Thefirst learning object may be recommended to a first student, while thesecond learning object may be recommended to a second student, eventhough the first and second student are cognitively similar with respectto personal attributes associated with mathematics.

Example Method for Dynamically Altering Learning Object RemediationPreferences

In an embodiment, a hierarchy of learning objects is maintained byskills hierarchy manager 180. Each learning object in the hierarchy isassociated with a corresponding skill and one or more correspondingassessment items. The assessment items measure the level of success thatusers attain with respect to the corresponding learning object. Inresponse to determining that a first user has attained a first level ofsuccess with respect to a first learning object, a second learningobject that is associated with a second skill is selected by learningexperience engine 110 for recommendation to the first user.

In response to determining that a second user has attained the firstlevel of success with respect to the first learning object, and based atleast in part on determining that the first user has attained a secondlevel of success with respect to the second learning object, learningexperience engine 110 selects a third learning object to recommend tothe second user, wherein the third learning object is associated with athird skill.

For example, a first student may have performed poorly on an assessmentfor a learning object designed to teach the user skills related tographing polynomial functions. To help the first student bolster hisskills, learning experience engine may recommend a second learningobject to the first student. The skill associated with the secondlearning object may be related to graphing functions generally. However,the first student may also perform poorly on an assessment for thesecond learning object. This may be an indication that the secondlearning object is not a good remediator for the first learning object,given the level of performance demonstrated by the first student.Therefore, when a second student achieves the same level of performanceon the assessment for the first learning object, the second studentreceives a recommendation for a third learning object, such as one thatfocuses on more general graphing skills.

Hardware Overview

According to one embodiment, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices or any other device that incorporates hard-wiredand/or program logic to implement the techniques.

For example, FIG. 5 is a block diagram that illustrates a computersystem 500 upon which an embodiment of the invention may be implemented.Computer system 500 includes a bus 502 or other communication mechanismfor communicating information, and a hardware processor 504 coupled withbus 502 for processing information. Hardware processor 504 may be, forexample, a general purpose microprocessor.

Computer system 500 also includes a main memory 506, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 502for storing information and instructions to be executed by processor504. Main memory 506 also may be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 504. Such instructions, when stored innon-transitory storage media accessible to processor 504, rendercomputer system 500 into a special-purpose machine that is customized toperform the operations specified in the instructions.

Computer system 500 further includes a read only memory (ROM) 508 orother static storage device coupled to bus 502 for storing staticinformation and instructions for processor 504. A storage device 510,such as a magnetic disk or optical disk, is provided and coupled to bus502 for storing information and instructions.

Computer system 500 may be coupled via bus 502 to a display 512, such asa cathode ray tube (CRT), for displaying information to a computer user.An input device 514, including alphanumeric and other keys, is coupledto bus 502 for communicating information and command selections toprocessor 504. Another type of user input device is cursor control 516,such as a mouse, a trackball, or cursor direction keys for communicatingdirection information and command selections to processor 504 and forcontrolling cursor movement on display 512. This input device typicallyhas two degrees of freedom in two axes, a first axis (e.g., x) and asecond axis (e.g., y), that allows the device to specify positions in aplane.

Computer system 500 may implement the techniques described herein usingcustomized hard-wired logic, one or more ASICs or FPGAs, firmware and/orprogram logic which in combination with the computer system causes orprograms computer system 500 to be a special-purpose machine. Accordingto one embodiment, the techniques herein are performed by computersystem 500 in response to processor 504 executing one or more sequencesof one or more instructions contained in main memory 506. Suchinstructions may be read into main memory 506 from another storagemedium, such as storage device 510. Execution of the sequences ofinstructions contained in main memory 506 causes processor 504 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “storage media” as used herein refers to any non-transitorymedia that store data and/or instructions that cause a machine tooperation in a specific fashion. Such storage media may comprisenon-volatile media and/or volatile media. Non-volatile media includes,for example, optical or magnetic disks, such as storage device 510.Volatile media includes dynamic memory, such as main memory 506. Commonforms of storage media include, for example, a floppy disk, a flexibledisk, hard disk, solid state drive, magnetic tape, or any other magneticdata storage medium, a CD-ROM, any other optical data storage medium,any physical medium with patterns of holes, a RAM, a PROM, and EPROM, aFLASH-EPROM, NVRAM, any other memory chip or cartridge.

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise bus 502. Transmission media can also take the formof acoustic or light waves, such as those generated during radio-waveand infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 504 for execution. For example,the instructions may initially be carried on a magnetic disk or solidstate drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 500 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 502. Bus 502 carries the data tomain memory 506, from which processor 504 retrieves and executes theinstructions. The instructions received by main memory 506 mayoptionally be stored on storage device 510 either before or afterexecution by processor 504.

Computer system 500 also includes a communication interface 518 coupledto bus 502. Communication interface 518 provides a two-way datacommunication coupling to a network link 520 that is connected to alocal network 522. For example, communication interface 518 may be anintegrated services digital network (ISDN) card, cable modem, satellitemodem, or a modem to provide a data communication connection to acorresponding type of telephone line. As another example, communicationinterface 518 may be a local area network (LAN) card to provide a datacommunication connection to a compatible LAN. Wireless links may also beimplemented. In any such implementation, communication interface 518sends and receives electrical, electromagnetic or optical signals thatcarry digital data streams representing various types of information.

Network link 520 typically provides data communication through one ormore networks to other data devices. For example, network link 520 mayprovide a connection through local network 522 to a host computer 524 orto data equipment operated by an Internet Service Provider (ISP) 526.ISP 526 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the“Internet” 528. Local network 522 and Internet 528 both use electrical,electromagnetic or optical signals that carry digital data streams. Thesignals through the various networks and the signals on network link 520and through communication interface 518, which carry the digital data toand from computer system 500, are example forms of transmission media.

Computer system 500 can send messages and receive data, includingprogram code, through the network(s), network link 520 and communicationinterface 518. In the Internet example, a server 530 might transmit arequested code for an application program through Internet 528, ISP 526,local network 522 and communication interface 518.

The received code may be executed by processor 504 as it is received,and/or stored in storage device 510, or other non-volatile storage forlater execution.

In the foregoing specification, embodiments of the invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. The specification and drawings are,accordingly, to be regarded in an illustrative rather than a restrictivesense. The sole and exclusive indicator of the scope of the invention,and what is intended by the applicants to be the scope of the invention,is the literal and equivalent scope of the set of claims that issue fromthis application, in the specific form in which such claims issue,including any subsequent correction.

1. A method, comprising: maintaining, at an online learning system, a hierarchy of learning objects, wherein each learning object of a plurality of learning objects in the hierarchy is associated with a corresponding skill and one or more content items associated with the corresponding skill; wherein a particular node in the hierarchy of learning objects is associated with a first skill and corresponds to both a first learning object and a second learning object, wherein: the first learning object is associated with a first content item that is associated with the first skill; and the second learning object is associated with a second content item that is associated with the first skill; receiving, from a first user of a plurality of users, a first learning recommendation request for a learning object associated with the first skill; generating, for the first user, a first learning recommendation, wherein the first learning recommendation identifies the first learning object; receiving, from a second user of the plurality of users, a second learning recommendation request for a learning object associated with the first skill; generating, for the second user, a second learning recommendation, wherein the second learning recommendation identifies the second learning object; wherein the method is performed by one or more computing devices.
 2. The method of claim 1, further comprising: maintaining for a plurality of users of an online education system, profiles, wherein the profile maintained for each of the plurality of users describes one or more education-related attributes associated with the corresponding user of the plurality of users; determining that the first learning object is associated with a greater level of success than the second learning object for users with a first value for a particular education-related attribute; determining that the second learning object is associated with a greater level of success than the first learning object for users with a second value for the particular education-related attribute.
 3. The method of claim 1, further comprising: determining that the first learning object is associated with a better overall success rate than the second learning object based at least in part on the success of the second user with respect to the second learning object; removing the second learning object from consideration as a learning object to be recommended to users.
 4. The method of claim 1, wherein generating the first learning recommendation comprises: in response to determining that the first user has previously interacted with the second learning object, selecting the first learning object as the learning object to recommend to the first user for the first skill.
 5. The method of claim 1, further comprising: selecting each learning object associated with the particular node for a learning recommendation at least once; based at least in part on a determination that the first learning object is associated with a better success rate than the second learning object, selecting the first learning object more frequently than the second learning object for learning recommendations.
 6. The method of claim 1, wherein: the second learning object is a potential learning object of a set of potential learning objects selected for consideration to be associated with the particular node; the method further comprises, in response to determining that the first learning object is associated with a greater level of success than the second learning object: associating the third learning object with the particular node; and removing the association between the second learning object and the particular node.
 7. The method of claim 6, wherein: in response to determining that the third learning object is associated with a greater level of success than the first learning object: associating a fourth learning object with the particular node; and removing the association between the first learning object and the particular node.
 8. A method, comprising: maintaining, at an online learning system, a plurality of learning objects, wherein each learning object of the plurality of learning objects is associated with a corresponding skill and one or more corresponding assessment items, wherein the one or more assessment items measure the level of success that users attain with respect to the skill associated with the corresponding learning object; in response to a first user attaining a first level of success with respect to the first skill associated with a first learning object, selecting a second learning object to recommend to the first user, wherein the second learning object is associated with a second skill and is located at a second position; in response to determining that a second user has attained the first level of success with respect to the first learning object, and based at least in part on determining that the first user has attained a second level of success with respect to the second learning object, selecting a third learning object to recommend to the second user, wherein the third learning object is associated with a third skill; wherein the method is performed by one or more computing devices.
 9. The method of claim 8, wherein: the plurality of objects are arranged in a hierarchy with one of: the second learning object is a child of the first learning object and the third learning object is a child of the second learning object; the second learning object is a parent of the first learning object and the third learning object is a parent of the second learning object; the second learning object is a first child of the first learning object and the third learning object is a second child of the first learning object; or the second learning object is a first parent of the first learning object and the third learning object is a second parent of the first learning object.
 10. The method of claim 8, further comprising: maintaining for the first and second users, profiles, wherein the profile maintained for each of the first and second users describes one or more education-related attributes associated with the corresponding user; wherein the third learning object is selected based at least in part on a determination that one or more education-related attributes of the second user is similar to one or more education-related attributes of the first user.
 11. The method of claim 8, further comprising: in response to detecting that the second user has achieved a second level of success with respect to the third learning object, selecting the first learning object to recommend to the second user.
 12. The method of claim 8, further comprising: in response determining that the first user has attained a second level of success with respect to the second learning object, storing a first remediation metric for the second learning object, wherein the first remediation metric provides a measurement of how successful the second learning object has been as a remediator.
 13. The method of claim 12, further comprising: maintaining for the first and second users, profiles, wherein the profile maintained for each of the first and second users describes one or more education-related attributes associated with the corresponding user; in response determining that the first user has attained a second level of success with respect to the second learning object, storing a second remediation metric for the second learning object, wherein the second remediation metric provides a measurement of how successful the second learning object has been as a remediator; wherein the first remediation metric is associated with a first education-related attribute and the second remediation metric is associated with a second education-related attribute.
 14. The method of claim 8, further comprising: in response to determining that the first user has previously interacted with a learning object that is associated with a first skill that is associated with the first learning object, performing one or more of the following: restricting the type of device that the user may use to interact with the third learning object; selecting a first tool for delivering content items to the user, wherein the first tool was not previously used for delivering content items to the user; or selecting a first content item to be delivered to the user, wherein the first content item was not previously delivered to the user.
 15. The method of claim 8, further comprising: in response to determining that a second user has attained the first level of success with respect to the first learning object, and based at least in part on determining that the first user has attained a second level of success with respect to the second learning object, selecting a fourth learning object to recommend to the second user, wherein the third learning object is associated with a fourth skill; wherein the third skill and the fourth skill are prerequisites to a first skill that is associated with the first learning object.
 16. A computer-readable non-transitory storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform: maintaining, at an online learning system, a hierarchy of learning objects, wherein each learning object of a plurality of learning objects in the hierarchy is associated with a corresponding skill and one or more content items associated with the corresponding skill; wherein a particular node in the hierarchy of learning objects is associated with a first skill and corresponds to both a first learning object and a second learning object, wherein: the first learning object is associated with a first content item that is associated with the first skill; and the second learning object is associated with a second content item that is associated with the first skill; receiving, from a first user of a plurality of users, a first learning recommendation request for a learning object associated with the first skill; generating, for the first user, a first learning recommendation, wherein the first learning recommendation identifies the first learning object; receiving, from a second user of the plurality of users, a second learning recommendation request for a learning object associated with the first skill; generating, for the second user, a second learning recommendation, wherein the second learning recommendation identifies the second learning object.
 17. The computer-readable non-transitory storage medium of claim 16, wherein the instructions further include instructions for: maintaining for a plurality of users of an online education system, profiles, wherein the profile maintained for each of the plurality of users describes one or more education-related attributes associated with the corresponding user of the plurality of users; determining that the first learning object is associated with a greater level of success than the second learning object for users with a first value for a particular education-related attribute; determining that the second learning object is associated with a greater level of success than the first learning object for users with a second value for the particular education-related attribute.
 18. The method of claim 16, wherein the instructions further include instructions for: determining that the first learning object is associated with a better overall success rate than the second learning object based at least in part on the success of the second user with respect to the second learning object; removing the second learning object from consideration as a learning object to be recommended to users.
 19. The computer-readable non-transitory storage medium of claim 16, wherein generating the first learning recommendation comprises: in response to determining that the first user has previously interacted with the second learning object, selecting the first learning object as the learning object to recommend to the first user for the first skill.
 20. The computer-readable non-transitory storage medium of claim 16, wherein the instructions further include instructions for: selecting each learning object associated with the particular node for a learning recommendation at least once; based at least in part on a determination that the first learning object is associated with a better success rate than the second learning object, selecting the first learning object more frequently than the second learning object for learning recommendations.
 21. The computer-readable non-transitory storage medium of claim 16, wherein: the second learning object is a potential learning object of a set of potential learning objects selected for consideration to be associated with the particular node; the instructions further include instructions for, in response to determining that the first learning object is associated with a greater level of success than the second learning object: associating the third learning object with the particular node; and removing the association between the second learning object and the particular node.
 22. The computer-readable non-transitory storage medium of claim 21, wherein: in response to determining that the third learning object is associated with a greater level of success than the first learning object: associating a fourth learning object with the particular node; and removing the association between the first learning object and the particular node.
 23. A computer-readable non-transitory storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform: maintaining, at an online learning system, a plurality of learning objects, wherein each learning object of the plurality of learning objects is associated with a corresponding skill and one or more corresponding assessment items, wherein the one or more assessment items measure the level of success that users attain with respect to the skill associated with the corresponding learning object; in response to a first user attaining a first level of success with respect to the first skill associated with a first learning object, selecting a second learning object to recommend to the first user, wherein the second learning object is associated with a second skill and is located at a second position; in response to determining that a second user has attained the first level of success with respect to the first learning object, and based at least in part on determining that the first user has attained a second level of success with respect to the second learning object, selecting a third learning object to recommend to the second user, wherein the third learning object is associated with a third skill.
 24. The computer-readable non-transitory storage medium of claim 23, wherein: the plurality of objects are arranged in a hierarchy with one of: the second learning object is a child of the first learning object and the third learning object is a child of the second learning object; the second learning object is a parent of the first learning object and the third learning object is a parent of the second learning object; the second learning object is a first child of the first learning object and the third learning object is a second child of the first learning object; or the second learning object is a first parent of the first learning object and the third learning object is a second parent of the first learning object.
 25. The computer-readable non-transitory storage medium of claim 23, wherein the instructions further include instructions for: maintaining for the first and second users, profiles, wherein the profile maintained for each of the first and second users describes one or more education-related attributes associated with the corresponding user; wherein the third learning object is selected based at least in part on a determination that one or more education-related attributes of the second user is similar to one or more education-related attributes of the first user.
 26. The computer-readable non-transitory storage medium of claim 23, wherein the instructions further include instructions for: in response to detecting that the second user has achieved a second level of success with respect to the third learning object, selecting the first learning object to recommend to the second user.
 27. The computer-readable non-transitory storage medium of claim 23, wherein the instructions further include instructions for: in response determining that the first user has attained a second level of success with respect to the second learning object, storing a first remediation metric for the second learning object, wherein the first remediation metric provides a measurement of how successful the second learning object has been as a remediator.
 28. The computer-readable non-transitory storage medium of claim 27, wherein the instructions further include instructions for: maintaining for the first and second users, profiles, wherein the profile maintained for each of the first and second users describes one or more education-related attributes associated with the corresponding user; in response determining that the first user has attained a second level of success with respect to the second learning object, storing a second remediation metric for the second learning object, wherein the second remediation metric provides a measurement of how successful the second learning object has been as a remediator; wherein the first remediation metric is associated with a first education-related attribute and the second remediation metric is associated with a second education-related attribute.
 29. The computer-readable non-transitory storage medium of claim 23, wherein the instructions further include instructions for: in response to determining that the first user has previously interacted with a learning object that is associated with a first skill that is associated with the first learning object, performing one or more of the following: restricting the type of device that the user may use to interact with the third learning object; selecting a first tool for delivering content items to the user, wherein the first tool was not previously used for delivering content items to the user; or selecting a first content item to be delivered to the user, wherein the first content item was not previously delivered to the user.
 30. The computer-readable non-transitory storage medium of claim 23, wherein the instructions further include instructions for: in response to determining that a second user has attained the first level of success with respect to the first learning object, and based at least in part on determining that the first user has attained a second level of success with respect to the second learning object, selecting a fourth learning object to recommend to the second user, wherein the third learning object is associated with a fourth skill; wherein the third skill and the fourth skill are prerequisites to a first skill that is associated with the first learning object. 